Methods and Systems for Creating a Volumetric Representation of a Real-World Event

ABSTRACT

An exemplary virtual reality media provider system differentiates static objects depicted in two-dimensional video data from dynamic objects depicted in the two-dimensional video data. Based on the differentiating of the static objects from the dynamic objects, the virtual reality media provider system generates dynamic volumetric models of the surfaces of the static objects and the dynamic objects. The virtual reality media provider system updates the dynamic volumetric models of the surfaces of the static objects with a lower regularity or on an as-needed basis, and the virtual reality media provider system separately updates the dynamic volumetric models of the surfaces of the dynamic objects with a higher regularity. The higher regularity is higher than the lower regularity and keeps the dynamic volumetric models of the surfaces of the dynamic objects up-to-date with what is occurring in the two-dimensional video data. Corresponding methods and systems are also disclosed.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 15/141,707, filed Apr. 28, 2016, and entitled“Methods and Systems for Creating and Providing a VolumetricRepresentation of a Real-World Event,” which is hereby incorporated byreference in its entirety.

BACKGROUND INFORMATION

Advances in computing and networking technology have made new forms ofmedia content possible. For example, virtual reality media content isavailable that may immerse viewers (or “users”) into interactive virtualreality worlds that the users may experience by directing theirattention to any of a variety of things being presented in the immersivevirtual reality world at the same time. For example, at any time duringthe presentation of the virtual reality media content, a userexperiencing the virtual reality media content may look around theimmersive virtual reality world in any direction with respect to both ahorizontal dimension (e.g., forward, backward, left, right, etc.) aswell as a vertical dimension (e.g., up, down, etc.), giving the user asense that he or she is actually present in and experiencing theimmersive virtual reality world from a particular viewpoint (e.g.,vantage point) within the immersive virtual reality world.

In some examples, a virtual reality media provider may provide virtualreality content that includes an immersive virtual reality worldrepresentative of a real-world event (e.g., a sporting event, a concert,etc.) that may be taking place in real time (i.e., a live event). Bytuning into a virtual reality broadcast of the real-world event, a usermay experience the real-world event by looking around the immersivevirtual reality world (e.g., the venue where real-world event is takingplace) at will during the real-world event. However, traditional virtualreality media content may limit the user to experiencing the immersivevirtual reality world from one or more static viewpoints within theimmersive virtual reality world.

For example, the user may be free to look around the immersive virtualreality world in any direction from one or more static locations at thereal-world event (e.g., static locations where creators of the virtualreality media content choose to position cameras capturing thereal-world event), but the user may be unable to experience theimmersive virtual reality world from other locations at the real-worldevent (e.g., locations where no camera is positioned). In many cases,some of the locations of most interest to users may be locations wherecameras cannot be positioned without interfering with the real-worldevent (e.g., on the field of a sporting event, on stage at a concert,etc.). Accordingly, the static viewpoints may limit the freedom of theuser to experience the real-world event from the most desirableviewpoints and/or may otherwise detract from the user experience in theimmersive virtual reality world.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate various embodiments and are a partof the specification. The illustrated embodiments are merely examplesand do not limit the scope of the disclosure. Throughout the drawings,identical or similar reference numbers designate identical or similarelements.

FIG. 1 illustrates an exemplary virtual reality media provider systemthat may create and provide a real-time volumetric representation of areal-world event according to principles described herein.

FIG. 2 illustrates an exemplary implementation of the virtual realitymedia provider system of FIG. 1 according to principles describedherein.

FIG. 3 illustrates an exemplary configuration in which the virtualreality media provider system of FIG. 1 operates to create a volumetricrepresentation of an exemplary real-world event according to principlesdescribed herein.

FIG. 4 illustrates exemplary media player devices configured tofacilitate a user in experiencing an immersive virtual reality worldbased on a volumetric representation of a real-world event, where thevolumetric representation of the real-world event is created andprovided by a virtual reality media provider system according toprinciples described herein.

FIG. 5 illustrates an exemplary configuration in which the virtualreality media provider system of FIG. 1 operates to create a volumetricrepresentation of an exemplary real-world event and provide time-shiftedvirtual reality media content representative of the real-world eventaccording to principles described herein.

FIG. 6 illustrates an exemplary configuration in which the virtualreality media provider system of FIG. 1 operates to create a volumetricrepresentation of an exemplary real-world event according to principlesdescribed herein.

FIG. 7 illustrates an exemplary technique for creating a dynamicvolumetric model of a surface of an exemplary object at an exemplaryreal-world event according to principles described herein.

FIG. 8 illustrates an exemplary dataflow for creating and providing areal-time volumetric representation of a real-world event according toprinciples described herein.

FIG. 9 illustrates an exemplary virtual reality experience in which auser is presented with exemplary virtual reality media contentrepresentative of a real-world event as experienced from a dynamicallyselectable viewpoint corresponding to an exemplary arbitrary location atthe real-world event according to principles described herein.

FIGS. 10 and 11 illustrate exemplary methods for creating and providinga real-time volumetric representation of a real-world event according toprinciples described herein.

FIG. 12 illustrates an exemplary computing device according toprinciples described herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Methods and systems for creating and providing a volumetricrepresentation of a real-world event are described herein. For example,the real-world event may be occurring live (e.g., in real time) and thevolumetric representation of the real-world event may be a real-timevolumetric data stream representative of a dynamic volumetric model ofthe surfaces of objects at the real-world event. As will be describedand illustrated below, a virtual reality media provider system mayinclude a configuration of synchronous video and depth capture devices(e.g., video cameras, three-dimensional (“3D”) depth scanning hardware,etc.) disposed at fixed positions at a real-world event. The real-worldevent may be any event that takes place in the real-world (i.e., asopposed to taking place only in a virtual world). For example, thereal-world event may be a sporting event (e.g., a basketball game, anOlympic event, etc.), a concert (e.g., a rock concert in a large venue,a classical chamber concert in an intimate venue, etc.), a theatricalpresentation (e.g., a Broadway musical, an outdoor pageant, etc.), alarge-scale celebration (e.g., New Year's Eve on Times Square, MardisGras, etc.), a race (e.g., a stock-car race, a horse race, etc.), apolitical event (e.g., a presidential debate, a political convention,etc.), or any other real-world event that may interest potentialviewers. The real-world event may take place at any indoor or outdoorreal-world location.

The configuration of synchronous video and depth capture devicesdisposed at the fixed positions at the real-world event may beconfigured to capture two-dimensional (“2D”) video data as well as depthdata for surfaces of objects at the real-world event. As used herein,“2D video data” may broadly include any data representative of how areal-world subject (e.g., a real-world scene, one or more objects at areal-world event, etc.) may appear over a particular time period andfrom at least one vantage point of at least one device capturing the 2Dvideo data. 2D video data is not limited to any particular format, filetype, frame rate, resolution, quality level, or other characteristicthat may be associated with various definitions and/or standardsdefining video in the art. In certain examples, 2D video data mayinclude a captured sequence of images (e.g., high-resolution stillimages) representative of an object at a real-world event over aparticular time period. As used herein, “depth data” may broadly includeany data representative of a position of a real-world subject (e.g., oneor more objects at a real-world event) in 3D space. As will be describedin more detail below, depth data may be captured based solely on 2Dvideo data (e.g., by combining 2D video data captured from differentvantage points using a suitable depth capture technique) or by usingtechniques that may require additional depth capture equipment and/ordata such as specialized depth capture devices that providetime-of-flight data, infrared imaging data, and the like. In certainexamples, 2D video data may be synchronized with depth data such thatdynamic volumetric models of the surfaces of objects that incorporatethe 2D video data and the depth data across a period of time may begenerated.

Accordingly, video and depth capture devices may capture 2D video dataand depth data in any suitable way and using any suitable devices as mayserve a particular implementation. Specifically, as will be described inmore detail below, in certain examples, video and depth capture devicemay consist of video cameras or other types of image capture devicesthat may capture 2D video data of objects at a real-world event frommultiple vantage points from which depth data for the surfaces of theobjects may be captured (e.g., derived) by using one or more depthcapture techniques (e.g., triangulation-based depth capture techniques)described herein. In other examples, as will also be described in moredetail below, video and depth capture devices may include video camerasor other types of image capture devices configured to capture the 2Dvideo data, as well as separate depth capture devices configured tocapture the depths of the surface of the objects using one or more ofthe depth capture techniques described below (e.g., time-of-flight-baseddepth capture techniques, infrared-based depth capture techniques,etc.). In the same or other examples, video and depth capture devicesmay include combination devices that include video camera devices andspecialized depth capture devices combined together in single devicesthat are similarly configured to capture the depth data using one ormore depth capture techniques described here. Additionally, theconfiguration of synchronous video and depth capture devices maycontinuously capture the 2D video data and the depth data in time, suchthat the surfaces of objects at the real-world event may be modeled inall four dimensions of space and time.

As used herein, an “object” may include anything that is visible (i.e.,non-transparent) from a particular viewpoint at a real-world event,whether living or inanimate. For example, as will be described below, ifthe real-world event is a basketball game, objects for whose surfacesthe video and depth capture devices may capture 2D video data and depthdata may include the basketball being used for the game, the basketballcourt, the basketball standards (i.e., the backboards, rims, nets,etc.), the players and referees participating in the game, and/or otherobjects present at and/or associated with the basketball game.

The video and depth capture devices may capture the 2D video data anddepth data in real-time (e.g., as the basketball game is being played)so that virtual reality media content representative of the real-worldevent (e.g., the basketball game) may be distributed to users toexperience live, as will be described below.

Based on the captured depth data and the captured 2D video data from thevideo and depth capture devices, the virtual reality media providersystem may generate a real-time volumetric data stream representative ofa dynamic volumetric model of the surfaces of the objects at thereal-world event. A dynamic volumetric model of an object may includeand/or be generated based both on 1) depth data representing where andhow the object is positioned in 3D space at a particular time, or withrespect to time over a particular time period, and on 2) synchronous 2Dvideo data mapped onto a positional model (e.g., a wireframe model ofthe object derived from the depth data) to represent how the objectappeared at the particular time or with respect to time over theparticular time period. As such, dynamic volumetric models may be 3Dmodels including three spatial dimensions or four-dimensional (“4D”)models that include the three spatial dimensions as well as a temporaldimension. The generation of the real-time volumetric data stream mayalso be performed in real time such that users not physically present atthe real-world event may be able to experience the real-world eventlive, in real time, via virtual reality media content representative ofthe real-world event. Examples of real-time volumetric data streams thatinclude dynamic volumetric models and techniques for creating anddistributing real-time volumetric data streams with dynamic volumetricmodels will be described below.

In some examples, the dynamic volumetric model of the surfaces of theobjects at the real-world event may be configured to be used to generatevirtual reality media content representative of the real-world event.The virtual reality media content may be generated by the virtualreality media provider system and/or by another system operated by thevirtual reality media provider or by a separate entity (e.g., a virtualreality media content distributor associated with the virtual realitymedia provider). Accordingly, virtual reality media content may begenerated and/or distributed (e.g., provided to one or more media playerdevices) by any suitable system (e.g., by the virtual reality mediaprovider system, a system associated with a virtual reality mediacontent distributor, etc.) based on the real-time volumetric data stream(i.e., based on the dynamic volumetric model of the surfaces of theobjects at the real world event). For example, the virtual reality mediaprovider system may provide the virtual reality media content to one ormore media player devices associated with respective users who may notbe physically present at the real-world event but who wish to experiencethe real-world event virtually using their media player devices. Asmentioned above, it may be desirable for the users who are not attendingthe real-world event to experience the real-world event live (e.g., inreal time as it is occurring with as small a delay as possible).Accordingly, the virtual reality media provider system may provide thevirtual reality media content representative of the real-world event tothe media player devices in real time.

While data processing and data distribution may take a finite amount oftime such that it is impossible for a user to experience real-worldevents precisely as the real-world events occurs, as used herein, anoperation (e.g., providing the virtual reality media content) isconsidered to be performed “in real time” when the operation isperformed immediately and without undue delay. Accordingly, a user maybe said to experience a real-world event in real time even if the userexperiences particular occurrences within the event (e.g., a particularshot in a basketball game) a few seconds or minutes after theoccurrences actually take place at the real-world event. Certain methodsand systems disclosed herein may be specially adapted to supportreal-time dynamic volumetric modeling and experiencing of immersivevirtual reality worlds based on live real-world events. For example,powerful hardware resources (e.g., multiple servers including multipleprocessing units) may be employed to perform the immense processingrequired for real-time creation and distribution of immersive virtualreality worlds based on real-time volumetric data streams representativeof dynamic volumetric models of the surfaces of objects at thereal-world event. Moreover, particular techniques for capturing 2D videodata and depth data (e.g., such as techniques described below) or fordistinguishing and separately modeling different types of objects (e.g.,static, dynamic, and background objects as described below) may furtherfacilitate and/or enable the immense processing to be performed inreal-time.

As mentioned above, it may be undesirable for a user experiencing areal-world event virtually (e.g., using a media player device to presentvirtual reality media content provided by a virtual reality mediaprovider system) to be limited to one or more discrete positions withinthe immersive virtual reality world representative of the real-worldevent. As such, the virtual reality media provider system may providethe virtual reality media content representative of the real-world eventas experienced from a dynamically selectable viewpoint corresponding toan arbitrary location at the real-world event. The dynamicallyselectable viewpoint may be selected by the user of the media playerdevice while the user is experiencing the real-world event using themedia player device.

As used herein, an “arbitrary location” may refer to any point in spaceat the real-world event. For example, arbitrary locations are notlimited to fixed positions where video and depth capture devices may bedisposed at the real-world event, but also include all the positionsbetween the video and depth capture devices and even places where videoand depth capture devices may not be able to be positioned. Moreover,arbitrary locations may not be limited to aligning with a viewing angle(i.e., an angle of capture) of any video and depth capture device in theconfiguration of synchronous video and depth capture device at thereal-world event. In some examples, such arbitrary locations (i.e., thatdo not directly align with a viewing angle of any video and depthcapture device) may correspond to the most desirable viewpoints at thereal-world event. For instance, in the basketball game example presentedabove, video and depth capture devices may not be allowed to bepositioned in the middle of the basketball court because the video anddepth capture devices would interfere with gameplay of the basketballgame.

In contrast, the user may dynamically select viewpoints from which toexperience the game that are in any arbitrary location on the basketballcourt. For example, the user may dynamically select his or her viewpointto follow the basketball up and down the basketball court and experiencethe basketball game as if standing on the basketball court in the middleof the action of the game. In other words, for example, while video anddepth capture devices may be positioned at fixed positions surroundingthe basketball court, but may not be positioned directly on the court soas not to interfere with gameplay of the basketball game, the user maydynamically select viewpoints from which to experience the game that arein any arbitrary location on the basketball court.

By creating and providing a real-time volumetric representation of areal-world event that allows users to dynamically select an arbitraryviewpoint from which to experience a real-world event as describedherein, a virtual reality media provider system may facilitate usersbecoming immersed in real-world events to an extent that may not bepossible for people watching the real-world events using traditionalmedia (e.g., television) or even experiencing the real-world eventsusing traditional virtual reality media. Moreover, the ability of usersto dynamically and arbitrarily move their viewpoint within thereal-world event may provide the users with an experience of thereal-world event not even available to physical attendees of thereal-world event. For example, users may be able to experience a livebasketball game as if running up and down the court with the players, orexperience a live concert as if standing on stage next to theperformers.

Various embodiments will now be described in more detail with referenceto the figures. The disclosed methods and systems may provide one ormore of the benefits mentioned above and/or various additional and/oralternative benefits that will be made apparent herein.

FIG. 1 illustrates an exemplary virtual reality media provider system100 (“system 100”) that may create and provide a real-time volumetricrepresentation of a real-world event in accordance with methods andsystems described herein for. As shown, system 100 may include, withoutlimitation, a data capture facility 102, a data processing facility 104,a data distribution facility 106, and a storage facility 108 selectivelyand communicatively coupled to one another. It will be recognized thatalthough facilities 102 through 108 are shown to be separate facilitiesin FIG. 1, facilities 102 through 108 may be combined into fewerfacilities, such as into a single facility, or divided into morefacilities as may serve a particular implementation. Each of facilities102 through 108 may include or be housed in a device (e.g., having asingle chassis) and located at a single location or distributed betweenmultiple devices and/or multiple locations as may serve a particularimplementation. Each of facilities 102 through 108 will now be describedin more detail.

Data capture facility 102 may include any hardware and/or software(e.g., computing systems, video and depth capture equipment, softwareprograms, etc.) used for capturing data associated with attributes ofobjects at a real-world event. For example, data capture facility 102may include a configuration of synchronous video and depth capturedevices such as 2D video cameras, 3D depth scanners, combinationvideo-depth capture devices (e.g., devices configured to capture both 2Dvideo and associated depth data), and so forth. Examples of video anddepth capture devices will be described in more detail below. Datacapture facility 102 may be used to capture two-dimensional video dataand depth data for surfaces of objects at a real-world event in any waydescribed herein and/or as may serve a particular implementation.

Data processing facility 104 may include any hardware and/or software(e.g., computing systems, software programs, etc.) used for processingthe data captured by data capture facility 102 and/or for generating areal-time volumetric data stream of a dynamic volumetric model of thesurfaces of the objects at the real-world event. For example, dataprocessing facility 104 may include one or more server systems or othercomputing devices running specialized and/or general-purpose imageprocessing software, 3D modeling software, and so forth. Examples of howdata processing facility 104 may process captured data and generate areal-time volumetric data stream based on the captured data will bedescribed below. Data processing facility 104 may also generate virtualreality media content representative of the real-world event based onthe real-time volumetric data stream.

Data distribution facility 106 may include any hardware and/or software(e.g., computing systems, networking systems, software programs, etc.)used for distributing data processed (e.g., generated) by dataprocessing facility 104 and/or for providing virtual reality mediacontent representative of the real-world event (e.g., virtual realitymedia content generated by data processing facility 104) as experiencedfrom a dynamically selectable viewpoint corresponding to an arbitrarylocation at the real-world event. To this end, data distributionfacility 106 may also receive data representative of user input (e.g.,selections of dynamically selectable viewpoints corresponding toarbitrary locations at the real-world event) from users experiencing thereal-world event using media player devices to present the virtualreality media content.

Storage facility 108 may maintain real-time volumetric data 110 and/orany other data received, generated, managed, maintained, used, and/ortransmitted by facilities 102 through 106. Real-time volumetric data 110may include data representing a dynamic volumetric model of the surfacesof the objects at the real-world event generated by data processingfacility 104 from 2D video data and/or depth data captured by datacapture facility 102. Real-time volumetric data 110 may include acomplete, real-time, volumetric (e.g., 3D) model of the real-world event(e.g., a four-dimensional model), any part of which may be presented toa user from any arbitrary viewpoint selected by the user. As such,system 100 may provide virtual reality media content representative ofthe real-world event as experienced from a dynamically selectableviewpoint corresponding to an arbitrary location at the real-world eventby providing different parts of real-time volumetric data 110 todifferent media player devices based on dynamically selectableviewpoints that are selected by different respective users of the mediaplayer devices. Storage facility 108 may further include any other dataas may be used by facilities 102 through 106 to create and provide thereal-time volumetric representation of the real-world event as may servea particular implementation.

FIG. 2 illustrates an exemplary implementation 200 of system 100 shownin FIG. 1. As shown, implementation 200 includes a configuration 202 ofsynchronous video and depth capture devices 204 (e.g., video and depthcapture devices 204-1 through 204-n). Implementation 200 furtherincludes a virtual reality media processing server 206 and a virtualreality media distribution server 208 communicatively coupled toconfiguration 202.

In configuration 202, synchronous video and depth capture devices 204(“capture devices 204”) may be disposed (i.e. located, installed, etc.)at fixed positions at a real-world event in any way that may serve aparticular implementation. For example, as will be illustrated anddescribed below, configuration 202 may include capture devices 204 atfixed positions surrounding a real-world event or one or more portionsthereof (e.g., surrounding a field of play of a sporting event such as abasketball court at a basketball game).

Each capture device 204 may include one or more devices or componentsconfigured to continuously capture 2D video and/or depth data as mayserve a particular implementation. For example, each capture device 204may include a first component (e.g., a video camera device) configuredto capture 2D video of objects at which the first component is directed(e.g., pointed), and a second component (e.g., a depth camera device, a3D imaging or 3D scanning device, etc.) configured to capture depth dataof objects at which the second component is directed. Is this example,the first component and the second component may be separate or discretedevices, but may be communicatively coupled and configured to work inconjunction with one another to simultaneously and synchronously captureboth the 2D video data and the depth data.

In other examples, each capture device 204 may comprise a combinationvideo-depth capture device (e.g., a specially-designed video camera)that is configured to capture both the 2D video data and the depth data.In other words, both the 2D video data and the depth data may becaptured using the same combination video-depth capture device. Thecombination video-depth capture device may be a commercially availableor specially-designed video camera capable of not only capturing videodata but also detecting corresponding depth of objects represented inthe video data using one of the depth capture techniques describedherein or another suitable technique. Similarly, as mentioned above, inexamples where a depth capture technique being used relies only on 2Dvideo data (e.g., certain triangulation-based depth capture techniques),capture devices 204 may not include any specialize depth captureequipment or capability (e.g., time-of-flight equipment, infraredsensing equipment, etc.) but, rather, may only include video capturedevices and/or other similar types of image capture devices.

In some examples, capture devices 204 may have a limited viewing angle(e.g., 90 degrees, 120 degrees, etc.) designed to capture data fromobjects at the real-world event in a specific area. For example, a ringconfiguration of capture devices 204 with limited viewing angles maysurround a real-world event or one or more portions thereof (e.g., abasketball court, turns on a racetrack) and be pointed inwardly tocapture objects at the real-world event (e.g., on the basketball court,the turns of the racetrack, etc.). In the same or other examples, atleast one particular capture device 204 may have a 360-degree viewingangle to capture data from objects surrounding the particular capturedevice 204. For example, at least one of capture devices 204 may be a360-degree camera configured to capture and/or generate a 360-degreevideo image of the real-world event around a center point correspondingto the 360-degree camera.

As used herein, a 360-degree video image is any video image that depictsthe surroundings of a center point (e.g., a center point associated withthe location of one of capture devices 204 such as a 360-degree camera)on all sides along at least one dimension. For example, one type of360-degree video image may include a panoramic video image that depictsa complete 360-degree by 45-degree ring around a center pointcorresponding to the camera. Another type of 360-degree video image mayinclude a spherical video image that depicts not only the ring aroundthe center point, but an entire 360-degree by 180-degree spheresurrounding the center point on all sides. In certain examples, a360-degree video image may be based on a non-circular geometricstructure. For example, certain 360-degree video images may be based oncubes, rectangular prisms, pyramids, and/or other geometric structuresthat may serve a particular implementation, rather than being based onspheres.

The 360-degree camera may be configured to capture a very wide-anglevideo image (e.g., using one or more “fish-eye” lenses to capture aspherical or semi-spherical image) or to capture a plurality of rawvideo images from each of a plurality of segment capture cameras builtinto or otherwise associated with the 360-degree camera. In someexamples, the 360-degree camera may generate the 360-degree video imageof the real-world event by combining (e.g., stitching together) theplurality of video images captured by the segment capture cameras. Inother examples, the 360-degree camera may send raw video image data toone or more servers (e.g., virtual reality media processing server 206)and the raw video images may be combined into a 360-degree (e.g.,spherical) video image by the one or more servers.

Capture devices 204 within configuration 202 may be communicativelycoupled to one another (e.g., networked together) and/or communicativelycoupled to another device (e.g., virtual reality media processing server206). This may allow the devices to maintain synchronicity in time,position, angle, etc. so that a dynamic volumetric model of the surfacesof the objects at the real-world event may be properly generated. Forexample, capture devices 204 may send and receive timing signals toensure that each of capture device 204 captures corresponding data atthe same time and that the data captured by different capture devices204 may be timestamped with a universal time shared by all of capturedevices 204 in configuration 202.

Virtual reality media processing server 206 may perform any of the dataprocessing operations described herein. For example, virtual realitymedia processing server 206 may be associated with (e.g., may implementall or a portion of or may be contained within) data processing facility104 and/or storage facility 108 of system 100. As such, virtual realitymedia processing server 206 may receive captured data from configuration202 of capture device 204 and may use the captured data to generate areal-time volumetric data stream representative of a dynamic volumetricmodel of the surfaces of the objects at the real-world event in any waythat may serve a particular implementation.

Virtual reality media distribution server 208 may perform any of thedata distribution operations described herein. For example, virtualreality media distribution server 208 may be associated with (e.g.,implementing all or a portion of, or being contained within) datadistribution facility 106 and/or storage facility 108 of system 100. Assuch, virtual reality media distribution server 208 may receive captureddata from configuration 202 and/or processed data (e.g., the real-timevolumetric data stream and/or virtual reality media content based on thereal-time volumetric data stream) from virtual reality media processingserver 206, and may distribute the captured and/or processed data toother devices. For example, virtual reality media distribution server208 may provide virtual reality media content representative of thereal-world event (e.g., based on the real-time volumetric data stream)to media player devices associated with users (not explicitly shown inFIG. 2).

FIG. 3 illustrates an exemplary configuration 300 in which system 100operates to create a volumetric representation of an exemplaryreal-world event. In the example of FIG. 3, the real-world event is abasketball game. As shown in configuration 300, a stage space 302 of thereal-world event (e.g., a basketball court) may be surrounded byinward-facing synchronous video and depth capture devices 304-i, and maysurround at least one outward-facing video and depth capture device304-o (collectively referred to as “capture devices 304”). Capturedevices 304 may be configured to capture 2D video data and depth data inreal time for surfaces of objects 306 at the real-world event (e.g.,players, the basketball, etc.). Basketball 308 is specifically calledout in configuration 300 as a particular example of an object 306because a detailed example of creating a dynamic volumetric model willbe provided below with respect to basketball 308.

As further shown in configuration 300, capture devices 304 may becommunicatively coupled by cables 310 and/or by other means (e.g.,wireless networking means) to one another and/or to one or morereal-time servers 312. Real-time servers 312, in turn arecommunicatively coupled by a network 314 to one or more media playerdevices associated with one or more respective users, including a mediaplayer device 316 associated with a user 318. Certain components inconfiguration 300 will now be described in more detail.

Stage space 302 may include any portion of a real-world event that istargeted by a virtual reality media provider as being of interest topotential virtual reality viewers (e.g., such as user 318). For example,if, as in the example of FIG. 3, the real-world event is a basketballgame, the real-world event may be the entire basketball arena where thegame is taking place (e.g., including the seating areas, etc.) whilestage space 302 may include only the basketball court itself and thespace above the basketball court where the game is played. In otherexamples, stage space 302 may include a stage where performers (e.g.,actors in a play, musicians at a concert, etc.) are performing, or otherrelevant areas of interest (e.g., specific turns and/or the finish lineon a racetrack) depending on the nature of the real-world event, thelevel of user interest in the real-world event, the financial resourcesand priorities of the virtual reality media provider capturing thereal-world event, and any other factors that may serve a particularimplementation.

In some examples, the fixed positions at the real-world event wherecapture devices 304 are disposed include fixed positions outside ofstage space 302 (e.g., off of the basketball court) while objects 306that capture devices 304 may be directed at and for which dynamicvolumetric models may be created and provided may be within stage space302 (e.g., on the basketball court). However, as described above, user318 may select dynamically selectable viewpoints at arbitrary locationswithin stage space 302 from which to experience the real-world event.

Capture devices 304 may be the same or similar to capture devices 204,described above in relation to FIG. 2. As shown, capture devices 304 maybe disposed at fixed positions at the real-world event such assurrounding stage space 302 (in the case of capture devices 304-i)and/or in the middle of stage space 302 (in the case of capture device304-o). Thus, as described above, capture devices 304-i may have limitedviewing angles but may be directed inward to continuously capturedetails of what is happening in stage space 302. Conversely, capturedevice 304-o may be a 360-degree outward facing synchronous video anddepth capture device (e.g., a 360-degree camera) configured tocontinuously capture 360-degree 2D video data and depth data forsurfaces of objects 306 within stage space 302, as well as for objects306 visible at the real-world event but that are outside of stage space302. For example, capture device 304-o may continuously capture datarepresentative of objects in the spectator seating areas at the venue inwhich the basketball game is taking place. Because capture device 304-omay not be able to be positioned directly within stage space 302 (i.e.,because it would interfere with the basketball game), capture device304-o may be suspended above stage space 302 or otherwise positioned asmay serve a particular implementation.

A configuration of capture devices 304 may include any suitable numberof cameras as may serve a particular implementation. For example, thenumber and position of capture devices 304 may be determined based on atarget quality level a virtual reality media provider strives to provideand/or based on a minimum number of cameras to reasonably capture datafrom objects 306 from enough angles to be able to adequately generatethe dynamic volumetric model of the surfaces of objects 306. In otherwords, even when objects 306 are dynamically moving around within stagespace 302 such that one object 306 may completely or partially block theview of another object 306 from the angle of a first capture device 304,the number and placement of capture devices 304 may ensure that a secondcapture device 304 will have a better angle with which to capture datafor the blocked object 306 than does the first capture device 304.

Objects 306 may include any objects at the real-world event inside oroutside stage space 302. For example, objects 306 may include people onthe court (e.g., basketball players, referees, and other people on thebasketball court), basketball 308, and/or other living and/or inanimateobjects such as basketball standards (i.e., backboards, rims, nets,etc.), the floor of the basketball court, people and/or furniture on thesidelines of the basketball game, spectators and seating areassurrounding the basketball court, and the like. A specific example ofhow 2D video data and depth data may be captured and used to generate areal-time volumetric data stream including a dynamic volumetric model ofbasketball 308 will be described below.

Real-time servers 312 may include any components described herein thatmay perform operations for creating and providing a real-time volumetricrepresentation of the basketball game real-world event. For example,real-time servers 312 may include a plurality of powerful server systems(e.g., having multiple graphics processing units) that implement system100 and/or any of the systems or facilities described in relation tosystem 100 in FIG. 1 or 2 or hereafter. In particular, real-time servers312 may receive captured data from capture devices 304 and generate areal-time volumetric data stream representative of a dynamic volumetricmodel of the surfaces of objects 306. Real-time servers 312 may thengenerate and provide virtual reality media content to media playerdevice 316 in real time (e.g., over network 314).

Network 314 may include any provider-specific wired or wireless network(e.g., a cable or satellite carrier network or a mobile telephonenetwork), the Internet, wide area network, or any other suitablenetwork. Data may flow between real-time servers 312, or betweenreal-time servers 312 and media player device 316 using anycommunication technologies, devices, media, and protocols as may serve aparticular implementation. For example, real-time servers 312 maycommunicate with one another or with media player device 316 using anysuitable communication technologies, devices, media, and/or protocolssupportive of data communications, including, but not limited to, socketconnections, Ethernet, data bus technologies, data transmission media,communication devices, Transmission Control Protocol (“TCP”), InternetProtocol (“IP”), File Transfer Protocol (“FTP”), Telnet, HypertextTransfer Protocol (“HTTP”), HTTPS, Session Initiation Protocol (“SIP”),Simple Object Access Protocol (“SOAP”), Extensible Mark-up Language(“XML”) and variations thereof, Real-Time Transport Protocol (“RTP”),User Datagram Protocol (“UDP”), Global System for Mobile Communications(“GSM”) technologies, Code Division Multiple Access (“CDMA”)technologies, Evolution Data Optimized Protocol (“EVDO”), 4G Long TermEvolution (“LTE”), Voice over IP (“VoIP”), Voice over LTE (“VoLTE”),WiMax, Time Division Multiple Access (“TDMA”) technologies, ShortMessage Service (“SMS”), Multimedia Message Service (“MMS”), radiofrequency (“RF”) signaling technologies, wireless communicationtechnologies (e.g., Bluetooth, Wi-Fi, etc.), in-band and out-of-bandsignaling technologies, and other suitable communications technologies.While only one network 314 is shown to interconnect real-time servers312 and media player device 316 in FIG. 3, it will be recognized thatthese devices and systems may intercommunicate by way of multipleinterconnected networks as may serve a particular implementation.

Media player device 316 may be used by user 318 to access and experiencevirtual reality media content received from system 100 (e.g., fromreal-time servers 312). To this end, media player device 316 may includeor be implemented by any device capable of presenting a field of view ofan immersive virtual reality world (e.g., an immersive virtual realityworld representative of the real-world event) and detecting user inputfrom user 318 to dynamically update a scene of the immersive virtualreality world presented within the field of view as user 318 experiencesthe immersive virtual reality world.

For example, the field of view may provide a window through which user318 may easily and naturally look around the immersive virtual realityworld. The field of view may be presented by media player device 316(e.g., on a display screen of media player device 316) and may includevideo depicting objects surrounding the user within the immersivevirtual reality world. Additionally, the field of view may dynamicallychange in response to user input provided by the user as the userexperiences the immersive virtual reality world. For example, the mediaplayer device may detect user input (e.g., moving or turning the displayscreen upon which the field of view is presented). In response, thefield of view may display different objects and/or objects seen from adifferent viewpoint (e.g., a viewpoint corresponding to the position ofthe display screen) in place of the objects seen from the previousviewpoint.

In some examples, media player device 316 may be configured to allowuser 318 to select respective virtual reality media content programs(e.g., associated with different live real-world events, as well asother types of virtual reality media content programs) that user 318 maywish to experience. In certain examples, media player device 316 maydownload virtual reality media content programs that user 318 mayexperience offline (e.g., without an active connection to real-timeservers 312). In other examples, media player device 316 may request andreceive data streams representative of virtual reality media contentprograms that user 318 experiences while media player device 316 remainsin active communication with real-time servers 312 (e.g., system 100) byway of network 314.

To facilitate user 318 in experiencing virtual reality media content,media player device 316 may include or be associated with at least onedisplay screen (e.g., a head-mounted display screen built into ahead-mounted virtual reality device or a display screen of a mobiledevice mounted to the head of the user with an apparatus such as acardboard apparatus) upon which scenes of an immersive virtual realityworld may be displayed. Media player device 316 may also includesoftware configured to receive, maintain, and/or process datarepresentative of the immersive virtual reality world to present thescenes of the immersive virtual reality world on the display screens ofthe media player devices. For example, media player device 316 mayinclude dedicated, standalone software applications (e.g., mobileapplications) configured to process and present data representative ofimmersive virtual reality worlds on the displays. In other examples, thesoftware used to present the particular scenes of the immersive virtualreality worlds may include non-dedicated software such as standard webbrowser applications.

Media player device 316 may take one of several different form factors.For example, media player device 316 may include or be implemented by ahead-mounted virtual reality device (e.g., a virtual reality gamingdevice) that includes a head-mounted display screen, by a personalcomputer device (e.g., a desktop computer, laptop computer, etc.), by amobile or wireless device (e.g., a smartphone, a tablet device, a mobilereader, etc.), or by any other device or configuration of devices thatmay serve a particular implementation to facilitate receiving and/orpresenting virtual reality media content. Different types of mediaplayer devices (e.g., head-mounted virtual reality devices, personalcomputer devices, mobile devices, etc.) may provide different types ofvirtual reality experiences having different levels of immersiveness foruser 318.

To illustrate, FIG. 4 shows different form factors of media playerdevice 316 configured to facilitate user 318 in experiencing animmersive virtual reality world based on a volumetric representation ofa real-world event created and provided to the media player devices bysystem 100 according to methods and systems described herein.

As one example, a head-mounted virtual reality device 402 may be mountedon the head of user 318 and arranged so that each of the eyes of user318 sees a distinct display screen 404 (e.g., display screens 404-1 and404-2) within head-mounted virtual reality device 402. In some examples,a single display screen 404 may be presented and shared by both eyes ofuser 318. In other examples, distinct display screens 404 withinhead-mounted virtual reality device 402 may be configured to displayslightly different versions of a field of view of an immersive virtualreality world (e.g., representative of the real-world event). Forexample, display screens 404 may be configured to display stereoscopicversions of the field of view that may be captured by one or morestereoscopic cameras to give user 318 the sense that the immersivevirtual reality world presented in the field of view isthree-dimensional. Display screens 404 may also be configured to fillthe peripheral vision of user 318, providing even more of a sense ofrealism to user 318.

Moreover, head-mounted virtual reality device 402 may include motionsensors (e.g., accelerometers), directional sensors (e.g.,magnetometers), orientation sensors (e.g., gyroscopes), and/or othersuitable sensors to detect natural movements (e.g., head movements) ofuser 318 as user 318 experiences the immersive virtual reality world.Thus, user 318 may provide input indicative of a desire to move thefield of view in a certain direction and by a certain amount in theimmersive virtual reality world by simply turning his or her head inthat direction and by that amount. User 318 may use a physical consoleor controller to dynamically select a dynamically selectable viewpointcorresponding to an arbitrary location at the real-world event (e.g., aviewpoint on stage space 302) from which to experience (e.g., lookaround) the immersive virtual reality world representative of thereal-world event.

As another example of a media player device 316, a personal computerdevice 406 having a display screen 408 (e.g., a monitor) may be used byuser 318 to experience the immersive virtual reality worldrepresentative of the real-world event. Because display screen 408 maynot provide the distinct stereoscopic view for each of the user's eyesand/or may not fill the user's peripheral vision, personal computerdevice 406 may not provide the same degree of immersiveness thathead-mounted virtual reality device 402 provides. However, personalcomputer device 406 may be associated with other advantages such as itsubiquity among casual virtual reality users that may not be inclined topurchase or use a head-mounted virtual reality device. In some examples,personal computer device 406 may allow a user to experience virtualreality content within a standard web browser so that user 318 mayconveniently experience the real-world event without using specialdevices or downloading special software. User 318 may provide user inputto personal computer device 406 by way of a keyboard 410, a mouse 412,and/or any other such input device as may serve a particularimplementation. For example, user 318 may use mouse 412 or navigationkeys on keyboard 410 to move the field of view (i.e., to look around theimmersive virtual reality world) and/or to dynamically select adynamically selectable viewpoint within the real-world event from whichto experience the real-world event (i.e., to “walk” or “fly” aroundwithin the immersive virtual reality world). In certain examples, acombination of keyboard 410 and mouse 412 may be used.

As yet another example of a media player device 316, a mobile device 414having a display screen 416 may be used by user 318 to experience theimmersive virtual reality world representative of the real-world event.Mobile device 414 may incorporate certain advantages of bothhead-mounted virtual reality devices and personal computer devices toprovide the most versatile type of media player device for experiencingthe immersive virtual reality world. Specifically, like personalcomputer devices, mobile devices are extremely ubiquitous, potentiallyproviding access to many more people than dedicated head-mounted virtualreality devices. However, because many mobile devices are equipped withmotion sensors, directional sensors, orientation sensors, etc., mobiledevices may also be configured to provide user 318 with an immersiveexperience comparable to that provided by head-mounted virtual realitydevices. For example, mobile device 414 may be configured to dividedisplay screen 416 into two versions (e.g., stereoscopic versions) of afield of view and to fill the peripheral vision of user 318 when mobiledevice 414 is mounted to the head of user 318 using a relativelyinexpensive and commercially-available mounting apparatus (e.g., acardboard apparatus). In other embodiments, mobile device 414 mayfacilitate experiencing the immersive virtual reality world by receivingmovement-based user input at arm's length (i.e., not mounted to the headof user 318 but acting as a hand-held dynamic window for experiencingthe immersive virtual reality world), by receiving swipe gestures on atouchscreen, or by other techniques that may serve a particularembodiment.

While examples of certain media player devices have been described, theexamples are illustrative and not limiting. A media player device mayinclude any suitable device and/or configuration of devices configuredto facilitate receipt and presentation of virtual reality media contentaccording to principles described herein. For example, a media playerdevice may include a tethered device configuration (e.g., a tetheredheadset device) or an untethered device configuration (e.g., a displayscreen untethered from a processing device). As another example, ahead-mounted virtual reality media player device or other media playerdevice may be used in conjunction with a virtual reality controller suchas a wearable controller (e.g., a ring controller) and/or a handheldcontroller.

As mentioned above, it may be desirable for user 318 to experience areal-world event in real time (e.g., at the same time the real-worldevent is occurring or after a trivial period of delay). However, incertain examples, user 318 may wish to experience the real-world eventin a time-shifted manner, rather than in real time. For example, if thereal-world event begins at a particular time and user 318 tunes in tothe virtual reality media content representative of the real-world eventfifteen minutes late, user 318 may wish to experience the real-worldevent starting from the beginning (i.e., starting fifteen minutes beforethe time that user 318 tunes into the virtual reality media contentrepresentative of the real-world event). Alternatively, user 318 may bebusy when the real-world event occurs and may wish to experience thereal-world event later (e.g., the following day).

To this end, system 100 may store and maintain, subsequent to providingvirtual reality media content representative of the real-world event inreal time, a recording of the real-time volumetric data streamrepresentative of the dynamic volumetric model of the surfaces of theobjects at the real-world event. Then, when user 318 later wishes toexperience the real-world event, system 100 may provide virtual realitymedia content representative of the real-world event as experienced froma dynamically selectable viewpoint corresponding to an arbitrarylocation at the real-world event selected by the user. For example, thevirtual reality media content may provide the virtual reality mediacontent to media player device 316 based on the recording of thereal-time volumetric data stream.

To illustrate, FIG. 5 shows an exemplary configuration 500 in whichsystem 100 operates to create a volumetric representation of areal-world event (e.g., a basketball game) and to provide time-shiftedvirtual reality media content representative of the real-world event tomedia player device 316. As shown, configuration 500 includes all of thesame elements discussed above in relation to configuration 300 of FIG.3, but further includes a time shift storage 502 component. Time shiftstorage 502 may be implemented within system 100 (e.g., within storagefacility 108). Additionally, while time shift storage 502 is illustratedas a stand-alone component in configuration 500, it will be understoodthat time shift storage 502 may be included within real-time servers 312and/or or within any other server or system as may serve a particularimplementation. When user 318 desires to experience a time-shifted,rather than a real time, version of the real-world event, system 100(e.g., real-time servers 312) may request and receive datarepresentative of the real-time volumetric data stream recorded in timeshift storage 502 and provide virtual reality media contentrepresentative of the real-world event to media player device 316 in thesame or a similar manner as if user 318 were experiencing the real-worldevent in real-time.

In certain examples, a real-world event may include a plurality of areas(e.g., geographical areas) of particular interest to users along withone or more areas of relatively less interest to users. As such, stagespace 302 may include several distinct (i.e., non-touching) parts, andthe dynamic volumetric model of the surfaces of the objects at thereal-world event may include a plurality of distinct volumetricsub-models each corresponding to an area at the real-world eventseparated from other areas at the real-world event (e.g., areas thatcorrespond to other volumetric sub-models of the dynamic volumetricmodel).

To illustrate, FIG. 6 shows an exemplary configuration 600 in whichsystem 100 operates to create a volumetric representation of anexemplary real-world event that includes a plurality of areas ofparticular interest. Specifically, as shown in configuration 600, thereal-world event may be a race (e.g., a stock car race) at a racetrack.At such a real-world event, it may be undesirable to set up a sufficientconfiguration of synchronous video and depth capture devices to generatea real-time volumetric data stream representative of a dynamicvolumetric model of all the surfaces of the objects around the entireracetrack (e.g., which may be several miles long). As a result, thevirtual reality media provider may select and designate several distinctand non-contiguous areas of interest at the real-world event (e.g.,particular curves of the racetrack, the finish line, etc.) as a stagespace 602.

For example, as shown in configuration 600, stage space 602 maycollectively refer to seven sub-stage spaces labeled in FIG. 6 as stagespaces 602-1 through 602-6 (corresponding to the six turns of theracetrack) and stage space 602-f (corresponding to the finish line). Aplurality of video and depth capture devices 604 (“capture devices 604”)may be positioned at fixed positions around the sub-stage space at eachturn of the racetrack (e.g., capture devices 604-1 around stage space602-1 at turn 1, capture devices 604-2 around stage space 602-2 at turn2, and similar capture devices around stage spaces 602-3 through 602-6at the other turns (not explicitly labeled in FIG. 6)) and around thesub-stage space at the finish line (e.g., capture device 604-f aroundstage space 602-f), and may be directed at the respective sub-stagespaces of stage space 602.

In FIG. 6, a plurality of objects 606 for whose surfaces 2D video dataand depth data are captured and used to generate a real-time volumetricdata stream may include stock cars racing around the racetrack, theracetrack itself, and other objects, scenery, and spectators around theracetrack. Real time servers 312 may receive the captured 2D video dataand depth data from capture devices 604 (e.g., via wired, not explicitlyshown in FIG. 6, or wireless communications, etc.) and process and storethe real-time volumetric data stream, as well as generate and providevirtual reality media content based on the real-time volumetric datastream in operation with time shift storage 502, network 314, and/ormedia player device 316, as described above.

An example will now be provided to illustrate how system 100 may capture2D video data and depth data for surfaces of an object at a real-worldevent and then generate a real-time volumetric data stream including adynamic volumetric model of the surfaces of the object at the real-worldevent. FIG. 7 illustrates an exemplary technique 700 for creating adynamic volumetric model of a surface of an exemplary object (e.g.,basketball 308 described above in relation to FIG. 3). As shown in FIG.7, a real-world event 702 may include an object 704. A configuration ofsynchronous video capture devices 706 and a configuration of synchronousdepth capture devices 708 may, respectively, capture 2D video data 710and depth data 712 for the surface of object 704 at real-world event702. For example, video capture devices 706 and depth capture devices708 may be disposed at fixed positions surrounding object 704 atreal-world event 702 such that 2D video data and depth data for theentire surface of object 704 (i.e., from every angle and vantage point)may be captured. 2D video data 710 and depth data 712 may then undergodynamic volumetric modeling 714 to generate a dynamic volumetric model716 (e.g., a 3D model) of object 704, which may be included with dynamicvolumetric models of the surfaces of other objects at the real-worldevent in a real-time volumetric data stream representative of thereal-world event.

Real-world event 702 may be any real-world event mentioned herein orthat may serve a particular embodiment. To continue with the examplepresented above in relation to FIGS. 3 and 5, for example, real-worldevent 702 may be a basketball game. Similarly, object 704 may be anyvisible (i.e. nontransparent) object mentioned herein or that mayotherwise be present at a real-world event. For example, object 704 maybe animate (e.g., a person or an animal) or inanimate, a solid, aliquid, or a non-transparent gas (e.g., fog generated from a fog machineat a concert), etc. In this example, as shown, object 704 is basketball308.

Video capture devices 706 and depth capture device 708 may be the sameor similar to other video and depth capture devices described herein,such as capture devices 204, 304, and/or 604. While only one videocapture device and one depth capture device is illustrated in FIG. 7, itwill be understood that each capture device may represent aconfiguration of capture devices that may surround object 704 to capturedata for the surface of object 704 from all sides (e.g., such as shownby capture devices 304 in FIG. 3). As shown, video capture devices 706and depth capture devices 708 may be standalone capture devices (e.g.,video cameras and 3D depth scanning devices, respectively).Alternatively, as described above, video capture devices 706 and depthcapture devices 708 may be integrated into combination video-depthcapture devices configured to capture both 2D video data and depth datausing the same combination video-depth capture devices. In someexamples, as mentioned above, depth data may be determined based solelyon 2D video data (e.g., 2D video data from different vantage points)such that depth capture devices 708 may represent the same video camerasand/or other types of image capture devices represented by video capturedevices 706.

2D video data 710 may be captured by video capture devices 706 and mayinclude image or texture data representative of visible characteristics(e.g., color, shading, surface texture, etc.) of the surface of object704 from all perspectives. For example, as illustrated in FIG. 7, 2Dvideo data 710 may capture data representative of visiblecharacteristics of various sections (e.g., small areas) of the surfaceof basketball 308 as the sections appear from various vantage points ofvarious video capture devices 706. For illustrative purposes, 2D videodata 710 in FIG. 7 shows a plurality of 2D images associated withvarious random sections of the surface of basketball 308 from a singlevantage point. However, it will be understood that 2D video data 710 mayinclude data associated with a plurality of vantage points surroundingbasketball 308 and may be captured, packaged, stored, formatted, andtransmitted in any way that may serve a particular embodiment. Forexample, 2D video data 710 may be delivered to dynamic volumetricmodeling 714 with detailed information (e.g., metadata) indicatingtemporal and spatial information, such as when the 2D video data wascaptured, where the 2D video data was captured, etc.

Similarly, depth data 712 may be captured by depth capture devices 708and may comprise depth data representative of spatial characteristics(e.g., locational coordinates, etc.) of the surface of object 704 fromall perspectives. For example, as illustrated in FIG. 7, depth data 712may include captured data representative of depth characteristics ofvarious sections (e.g., small areas) of the surface of basketball 308such that a wireframe model of basketball 308 may be generated (e.g.,stitched together) based on the depth data captured from various vantagepoints associated with each depth capture device 708. Depth data 712 maybe captured, packaged, stored, formatted, and transmitted in any waythat may serve a particular embodiment. For example, depth data 712 maybe delivered to dynamic volumetric modeling 714 with detailedinformation (e.g., metadata) indicating temporal and spatialinformation, such as when the depth data was captured, where the depthdata was captured, etc.

Depth data 712 may be determined by depth capture devices 708 using anytechnique or modality that may serve a particular implementation. Inparticular, certain depth capture techniques may be used to increase thetime efficiency of the depth capture (i.e., by minimizing capture and/orprocessing time) to facilitate generating dynamic volumetric models inreal time.

For example, depth capture devices 708 may capture depth data 712 byusing a stereoscopic triangulation depth capture technique. In thistechnique, depth capture devices 708 may be configured to capture 2Dvideo data (i.e., depth capture devices 708 may be one and the same asvideo capture devices 706). The stereoscopic triangulation depth capturetechnique may include a first depth capture device 708 capturing 2Dvideo data of points on the surface of object 704 from a first angle anda second depth capture device 708 capturing 2D video data of the pointson the surface of object 704 from a second angle. The depth of thepoints on the surface of object 704 are triangulated based on the firstangle, the second angle, and on a predetermined distance (i.e., a knowndistance based on the configuration of depth capture devices 708)between the first depth capture device 708 and the second depth capturedevice 708.

In the same or other examples, depth capture devices 708 may capturedepth data 712 by using a time-of-flight depth capture technique. Forexample, depth capture devices 708 may use a radar-based rangingtechnique (e.g., laser radar) using electromagnetic pulses, asonar-based ranging technique using sound pulses, and/or any other typeof ranging technique as may serve a particular implementation. In thetime-of-flight technique, each depth capture device 708 may generate apulse (e.g., an electromagnetic pulse, a sound pulse, etc.) from asource associated with the depth capture device 708 at a particulartime, and may be specially configured to measure a total transit timefor the pulse to travel from the pulse source to points on the surfaceof object 704 (i.e., to travel to object 704), and, after beingreflected by the surface of object 704, to travel from the points on thesurface of object 704 to a pulse detector associated with the depthcapture device 708 (i.e., to return back to the depth capture device708). Based on the total transit time and the known speed of the pulse(e.g., the speed of light, the speed of sound, etc.), a depth of each ofthe points on the surface of object 704 may thus be determined.

In the same or other examples, depth capture devices 708 may capturedepth data 712 by using an infrared pattern analysis depth capturetechnique. In this technique, an infrared pattern emitter device (i.e.,associated with or separate from depth capture devices 708) may projecta random scatter (i.e., a pattern) of randomly-sized infrared dots ontosurfaces of various objects at real-world event 702, including object704. A first depth capture device 708 may be configured with infraredsensing capability such that the first depth capture device 708 maydetect the random scatter of randomly-sized infrared dots projected ontothe surfaces of the objects from a first angle. Similarly, a seconddepth capture device 708 similarly configured with infrared sensingcapability may detect the random scatter of randomly-sized infrared dotsprojected onto the surfaces of the objects from a second angle. Thedepth of the surfaces of the objects may then be triangulated based onthe first angle, the second angle, and on a predetermined distance(i.e., a known distance based on the configuration of depth capturedevices 708) between the first depth capture device 708 and the seconddepth capture device 708.

Because real-time depth detection in a non-controlled, real-worldenvironment may be difficult and inexact, in some examples, a pluralityof different depth capture techniques may be employed (e.g., such as thedepth capture techniques described above). Subsequently, depth dataobtained using each of the depth capture techniques employed may becombined to determine the most accurate depth data for objects withinreal-world event 702 possible.

Once 2D video data 710 and depth data 712 have been captured, dynamicvolumetric modeling 714 may process 2D video data 710 together withdepth data 712 to generate dynamic volumetric model 716 of object 704 inany suitable way. For example, dynamic volumetric modeling 714 maycombine depth data 712 (e.g., using temporal and spatial metadataincluded with depth data 712) to create (e.g., stitch together) awireframe model of basketball 308, as shown in the drawing representingdepth data 712. Dynamic volumetric modeling 714 may then map 2D videodata 710 onto the wireframe model by matching temporal and spatialmetadata included with 2D video data 710 with the temporal and spatialinformation included with depth data 712.

In this way, dynamic volumetric modeling 714 may generate dynamicvolumetric model 716 (e.g., a 3D model) of object 704 from 2D video data710 and depth data 712. Dynamic volumetric model 716 may be includedwith dynamic volumetric models of the surfaces of other objects atreal-world event 702 in a real-time volumetric data streamrepresentative of real-world event 702. However, each dynamic volumetricmodel of each object within real-world event 702 may be individually andindependently manipulable (e.g., processable) in relation to the otherdynamic volumetric models of the other objects at real-world event 702.Accordingly, based on the real-time volumetric data stream includingdynamic volumetric model 716 and other dynamic volumetric models forother objects at real-world event 702, virtual reality media content maybe generated and provided such that a user may view real-world event 702in real time from any arbitrary location at real-world event 702. Bygenerating distinct, individually-manipulable volumetric models of eachobject within real-world event 702 by combining depth data to generatewireframe models of individual objects at specific points in space atreal-world event 702 and mapping 2D video data onto the wireframe modelsto create dynamic volumetric models of the objects at real-world event702, virtual reality media content may be generated and provided asexperienced from viewpoints of real-world event 702 that may not bepossible (or may be extremely inefficient and/or processing intensive)by combining 2D video data alone.

FIG. 8 illustrates an exemplary dataflow 800 for creating and providinga real-time volumetric representation of a real-world event (e.g.,real-world event 702). The data in dataflow 800 may be generated,processed, distributed, etc., in any way described herein or as mayserve a particular implementation. As shown in FIG. 8, 2D video-depthdata 802 (e.g., 2D video-depth data 802-1 through 802-n and 802-o) mayflow into dynamic volumetric modeling 714, where static object depthmodeling 804, static object image mapping 806, dynamic object depthmodeling 808, dynamic object image mapping 810, and external modeling812 may process 2D video-depth data 802 to generate a real-timevolumetric data stream 814. In real time, virtual reality media content816, which may be generated based on real-time volumetric data stream814, may then be provided to a media player device.

2D video-depth data 802 may represent captured 2D video data andcaptured depth data from a plurality of video and depth capture devicessuch as capture devices 204 (see FIG. 2), capture devices 304 (see FIGS.3 and 5), capture devices 604 (see FIG. 6), or capture devices 706 and708 (see FIG. 7). For example, 2D video-depth data 802-1 may include 2Dvideo data (e.g., similar to 2D video data 710) and depth data (e.g.,similar to depth data 712) captured by a first video and depth capturedevice, 2D video-depth data 802-2 may include 2D video data and depthdata captured by a second video and depth capture device (e.g., a videoand depth capture device capturing data representative of objects from adifferent vantage point than the first video and depth capture device),and so forth for 2D video-depth data 802-3 through 802-n. 2D video-depthdata 802-o may include 2D video data and/or depth data captured by anoutward facing video and depth capture device (e.g., a 360-degreeoutward facing synchronous video and depth capture device such ascapture device 304-o in FIG. 3).

As described above in relation to FIG. 7, dynamic volumetric modeling714 may perform data processing on 2D video-depth data 802 to generate areal-time volumetric data stream representative of a dynamic volumetricmodel of the surfaces of the objects at the real-world event. Morespecifically, dynamic volumetric modeling 714 may generate a dynamicvolumetric model of the surfaces of at least three categories ofobjects: 1) static objects at the real-world event (e.g., a floor of abasketball court, basketball standards, etc.), 2) dynamic objects at thereal-world event (e.g., players and referees moving around on thebasketball court, a basketball being used by the players in a basketballgame, etc.), and 3) external objects at the real-world event (e.g.,objects outside the basketball court stage space such as spectatorswatching the basketball game, etc.). As mentioned above and as will bedescribed in more detail below, system 100 may obtain significantefficiency gains by differentiating these categories of objects andgenerating a real-time volumetric data stream representative of dynamicvolumetric models of the surfaces of the objects at the real-world eventseparately, rather than treating the different categories of objectsequally. For example, by differentiating static, dynamic, and externalobjects as described below, system 100 may obtain efficiency gains thatfacilitate and/or enable system 100 to perform the immense processingrequired to generate and provide the real-time volumetric data streamrepresentative of the dynamic volumetric models of the surfaces of theobjects at the real-world event in real time.

Static object depth modeling 804 may model (e.g., create wireframe depthmodels) one or more static objects at the real-world event based ondepth data within 2D video-depth data 802. For example, static objectdepth modeling 804 may determine, based on depth data, that a basketballstandard is statically located at a particular location in the spaceabove the basketball court, that the basketball standard is distinctfrom players and/or a basketball that may occasionally touch thebasketball standard, and that the basketball standard is distinct fromother objects seen behind the basketball standard (e.g., in thebackground) when the basketball standard is viewed from differentvantage points. With these determinations, static object depth modeling804 may generate a depth model (e.g., a wireframe model) of thebasketball standard that may not yet include any color or video data,but that may represent a location in a 3D space representative of thereal-world event where the basketball standard is positioned.

Static object image mapping 806 may map textures, colors, etc. onto thedepth model of static objects (e.g., the basketball standard) generatedby static object depth modeling 804. For example, static object imagemapping 806 may map the textures, colors, and so forth based on 2D videodata within 2D video-depth data 802. As such, complete dynamicvolumetric models of the static objects may be included within real-timevolumetric data stream 814. Because the static objects may not changeoften or at all, dynamic volumetric models of the static objects may beprocessed and updated irregularly or on an as-needed basis in order toconserve processing resources in system 100.

Dynamic object depth modeling 808 and dynamic object image mapping 810may perform similar respective functions as static object depth modeling804 and static object image mapping 806 for dynamic objects (e.g.,objects determined to be dynamically moving in real time). However,because the dynamic object may be continuously in flux (e.g., movingaround within the stage space of the real-world event), dynamicvolumetric models of the dynamic objects may be updated much moreregularly in order to keep the real-time volumetric data streamup-to-date with what is occurring at the real-world event.

External modeling 812 also may perform similar functions as the depthmodeling and image mapping operations described above for externalobjects (e.g., background objects that are not within the stage space)such as those represented in 2D video-depth data 802-o. Because theexternal objects may add ambience and realism to a virtual realityexperience but may not be a primary focus of the experience for manyusers, external modeling 812 may update models for external objectsirregularly. Additionally, because 2D video-depth data 802-o may includecaptured data from only one or a limited number of vantage points (i.e.,vantage points that do not surround the external objects to capture datafrom every vantage point), external modeling 812 may generate a 2D model(e.g., that incorporates little or no depth data but is just based on 2Dvideo data) of the external objects, or a volumetric model that includesless detail than the volumetric models of, for example, the dynamicobjects within the stage space of the real-world event.

Real-time volumetric data stream 814 may include models (e.g., dynamicvolumetric models, 2D models, etc.) of each of the static objects,dynamic objects, and external objects at the real-world event. Putanother way, real-time volumetric data stream 814 may be representativeof a dynamic volumetric model of the surfaces of some or all of theobjects at the real-world event. Accordingly, virtual reality mediacontent 816 may be provided based on real-time volumetric data stream814 to present a dynamically selectable viewpoint of the real-worldevent corresponding to an arbitrary location (e.g., an arbitrarylocation within the stage space) at the real-world event.

To illustrate, FIG. 9 shows an exemplary virtual reality experience 900in which user 318 is presented with exemplary virtual reality mediacontent 902 representative of a real-world event as experienced from adynamically selectable viewpoint corresponding to an exemplary arbitrarylocation at the real-world event. Specifically, virtual reality mediacontent 902 is presented within a field of view 904 that shows thereal-world event from a viewpoint corresponding to an arbitrary locationright underneath a basketball standard at the real-world event where ashot is being made. An immersive virtual reality world 906 based on thereal-world event may be available for the viewer to experience byproviding user input (e.g., head movements, keyboard input, etc.) tolook around and/or to move around (i.e., dynamically select a viewpointfrom which to experience) immersive virtual reality world 906.

In FIG. 9, immersive virtual reality world 906 is illustrated as asemi-sphere, indicating that user 318 may look in any direction withinimmersive virtual reality world 906 that is substantially forward,backward, left, right, and/or up from the viewpoint of the locationunder the basketball standard that user 318 has currently selected. Inother examples, immersive virtual reality world 906 may include anentire 360-degree by 180-degree sphere such that user 318 may also lookdown. Additionally, user 318 may move around to other locations withinimmersive virtual reality world 906 (i.e., dynamically selectingdifferent dynamically selectable viewpoints of the real-world event).For example, user 318 may select a viewpoint at half court, a viewpointfrom the free-throw line facing the basketball standard, a viewpointsuspended above the basketball standard, or the like.

FIG. 10 illustrates an exemplary method 1000 for creating and providinga real-time volumetric representation of a real-world event. While FIG.10 illustrates exemplary operations according to one embodiment, otherembodiments may omit, add to, reorder, and/or modify any of theoperations shown in FIG. 10. One or more of the operations shown in FIG.10 may be performed by system 100 and/or any implementation thereof.

In operation 1002, a virtual reality media provider system that includesa configuration of synchronous video and depth capture devices disposedat fixed positions at a real-world event may capture 2D video data anddepth data for surfaces of objects at the real-world event. In someexamples, the virtual reality media provider system may capture the 2Dvideo data and depth data in real time. Operation 1002 may be performedin any of the ways described herein.

In operation 1004, the virtual reality media provider system maygenerate a real-time volumetric data stream representative of a dynamicvolumetric model of the surfaces of the objects at the real-world event.In some examples, the virtual reality media provider system may generatethe real-time volumetric data stream in real time based on the captureddepth data and the captured two-dimensional video data captured inoperation 1002. Additionally, in certain examples, the dynamicvolumetric model of the surfaces of the objects at the real-world eventmay be configured to be used to generate virtual reality media contentrepresentative of the real-world event as experienced from a dynamicallyselectable viewpoint corresponding to an arbitrary location at thereal-world event, the dynamically selectable viewpoint selected by auser of a media player device while the user is experiencing thereal-world event using the media player device. Operation 1004 may beperformed in any of the ways described herein.

In operation 1006, the virtual reality media provider system may providevirtual reality media content representative of the real-world event toa media player device associated with a user. In some examples, theprovided virtual reality media content may be representative of thereal-world event as experienced from a dynamically selectable viewpointcorresponding to an arbitrary location at the real-world event. Thedynamically selectable viewpoint may be selected by a user of the mediaplayer device while the user is experiencing the real-world event usingthe media player device. The virtual reality media provider system mayprovide the virtual reality media content in real time based on thereal-time volumetric data stream generated in operation 1004. Operation1006 may be performed in any of the ways described herein. For example,operation 1006 may be performed by the virtual reality media providersystem or by a separate system (e.g., by another system operated by thevirtual reality media provider or by a virtual reality media contentdistributor associated with the virtual reality media provider) thatreceives the virtual reality media content from the virtual realitymedia provider system and/or distributes the virtual reality mediacontent under direction of the virtual reality media provider system.

FIG. 11 illustrates an exemplary method 1100 for creating and providinga real-time volumetric representation of a real-world event. While FIG.11 illustrates exemplary operations according to one embodiment, otherembodiments may omit, add to, reorder, and/or modify any of theoperations shown in FIG. 11. One or more of the operations shown in FIG.11 may be performed by system 100 and/or any implementation thereof.

In operation 1102, a virtual reality media provider system that includesa configuration of synchronous video and depth capture devices disposedat fixed positions surrounding a stage space where a real-world event istaking place and outside of a boundary around the stage space maycapture 2D video data and depth data for surfaces of objects locatedinside the boundary and associate with the real-world event taking placewithin the stage space. In some examples, the virtual reality mediaprovider system may capture the 2D video data and depth data in realtime. Operation 1102 may be performed in any of the ways describedherein.

In operation 1104, the virtual reality media provider system maygenerate a real-time volumetric data stream representative of a dynamicvolumetric model of the surfaces of the objects located inside theboundary around the stage space. For example, the virtual reality mediaprovider system may generate the real-time volumetric data stream inreal time based on the captured depth data and the capturedtwo-dimensional video data captured in operation 1102. Operation 1104may be performed in any of the ways described herein.

In operation 1106, the virtual reality media provider system may providevirtual reality media content representative of the real-world event toa media player device. For example, the virtual reality media contentmay be representative of the real-world event as experienced from adynamically selectable viewpoint corresponding to an arbitrary locationwithin the boundary around the stage space where the real-world event istaking place. The dynamically selectable viewpoint may be selected by auser of the media player device while the real-world event is takingplace and the user is experiencing the real-world event using the mediaplayer device. In some examples, operation 1106 may be performed in realtime based on the real-time volumetric data stream generated inoperation 1104. Operation 1106 may be performed in any of the waysdescribed herein.

In certain embodiments, one or more of the systems, components, and/orprocesses described herein may be implemented and/or performed by one ormore appropriately configured computing devices. To this end, one ormore of the systems and/or components described above may include or beimplemented by any computer hardware and/or computer-implementedinstructions (e.g., software) embodied on at least one non-transitorycomputer-readable medium configured to perform one or more of theprocesses described herein. In particular, system components may beimplemented on one physical computing device or may be implemented onmore than one physical computing device. Accordingly, system componentsmay include any number of computing devices, and may employ any of anumber of computer operating systems.

In certain embodiments, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices. In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein. Such instructions may be stored and/or transmittedusing any of a variety of known computer-readable media.

A computer-readable medium (also referred to as a processor-readablemedium) includes any non-transitory medium that participates inproviding data (e.g., instructions) that may be read by a computer(e.g., by a processor of a computer). Such a medium may take many forms,including, but not limited to, non-volatile media, and/or volatilemedia. Non-volatile media may include, for example, optical or magneticdisks and other persistent memory. Volatile media may include, forexample, dynamic random access memory (“DRAM”), which typicallyconstitutes a main memory. Common forms of computer-readable mediainclude, for example, a disk, hard disk, magnetic tape, any othermagnetic medium, a compact disc read-only memory (“CD-ROM”), a digitalvideo disc (“DVD”), any other optical medium, random access memory(“RAM”), programmable read-only memory (“PROM”), electrically erasableprogrammable read-only memory (“EPROM”), FLASH-EEPROM, any other memorychip or cartridge, or any other tangible medium from which a computercan read.

FIG. 12 illustrates an exemplary computing device 1200 that may bespecifically configured to perform one or more of the processesdescribed herein. As shown in FIG. 12, computing device 1200 may includea communication interface 1202, a processor 1204, a storage device 1206,and an input/output (“I/O”) module 1208 communicatively connected via acommunication infrastructure 1210. While an exemplary computing device1200 is shown in FIG. 12, the components illustrated in FIG. 12 are notintended to be limiting. Additional or alternative components may beused in other embodiments. Components of computing device 1200 shown inFIG. 12 will now be described in additional detail.

Communication interface 1202 may be configured to communicate with oneor more computing devices. Examples of communication interface 1202include, without limitation, a wired network interface (such as anetwork interface card), a wireless network interface (such as awireless network interface card), a modem, an audio/video connection,and any other suitable interface.

Processor 1204 generally represents any type or form of processing unitcapable of processing data or interpreting, executing, and/or directingexecution of one or more of the instructions, processes, and/oroperations described herein. Processor 1204 may direct execution ofoperations in accordance with one or more applications 1212 or othercomputer-executable instructions such as may be stored in storage device1206 or another computer-readable medium.

Storage device 1206 may include one or more data storage media, devices,or configurations and may employ any type, form, and combination of datastorage media and/or device. For example, storage device 1206 mayinclude, but is not limited to, a hard drive, network drive, flashdrive, magnetic disc, optical disc, RAM, dynamic RAM, other non-volatileand/or volatile data storage units, or a combination or sub-combinationthereof. Electronic data, including data described herein, may betemporarily and/or permanently stored in storage device 1206. Forexample, data representative of one or more executable applications 1212configured to direct processor 1204 to perform any of the operationsdescribed herein may be stored within storage device 1206. In someexamples, data may be arranged in one or more databases residing withinstorage device 1206.

I/O module 1208 may include one or more I/O modules configured toreceive user input and provide user output. One or more I/O modules maybe used to receive input for a single virtual reality experience. I/Omodule 1208 may include any hardware, firmware, software, or combinationthereof supportive of input and output capabilities. For example, I/Omodule 1208 may include hardware and/or software for capturing userinput, including, but not limited to, a keyboard or keypad, atouchscreen component (e.g., touchscreen display), a receiver (e.g., anRF or infrared receiver), motion sensors, and/or one or more inputbuttons.

I/O module 1208 may include one or more devices for presenting output toa user, including, but not limited to, a graphics engine, a display(e.g., a display screen), one or more output drivers (e.g., displaydrivers), one or more audio speakers, and one or more audio drivers. Incertain embodiments, I/O module 1208 is configured to provide graphicaldata to a display for presentation to a user. The graphical data may berepresentative of one or more graphical user interfaces and/or any othergraphical content as may serve a particular implementation.

In some examples, any of the facilities described herein may beimplemented by or within one or more components of computing device1200. For example, one or more applications 1212 residing within storagedevice 1206 may be configured to direct processor 1204 to perform one ormore processes or functions associated with data capture facility 102,data processing facility 104, or data distribution facility 106 ofsystem 100 (see FIG. 4). Likewise, storage facility 108 of system 100may be implemented by or within storage device 1206.

To the extent the aforementioned embodiments collect, store, and/oremploy personal information provided by individuals, it should beunderstood that such information shall be used in accordance with allapplicable laws concerning protection of personal information.Additionally, the collection, storage, and use of such information maybe subject to consent of the individual to such activity, for example,through well known “opt-in” or “opt-out” processes as may be appropriatefor the situation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

In the preceding description, various exemplary embodiments have beendescribed with reference to the accompanying drawings. It will, however,be evident that various modifications and changes may be made thereto,and additional embodiments may be implemented, without departing fromthe scope of the invention as set forth in the claims that follow. Forexample, certain features of one embodiment described herein may becombined with or substituted for features of another embodimentdescribed herein. The description and drawings are accordingly to beregarded in an illustrative rather than a restrictive sense.

What is claimed is:
 1. A method comprising: differentiating, by avirtual reality media provider system, a static object depicted intwo-dimensional video data from a dynamic object depicted in thetwo-dimensional video data; generating, by the virtual reality mediaprovider system and based on the differentiating of the static objectfrom the dynamic object, a dynamic volumetric model of a surface of thestatic object and a dynamic volumetric model of a surface of the dynamicobject; updating, by the virtual reality media provider system and witha lower regularity or on an as-needed basis, the dynamic volumetricmodel of the surface of the static object; and separately updating, bythe virtual reality media provider system and with a higher regularitythat is higher than the lower regularity and that keeps the dynamicvolumetric model of the surface of the dynamic object up-to-date withwhat is occurring in the two-dimensional video data, the dynamicvolumetric model of the surface of the dynamic object.
 2. The method ofclaim 1, wherein the two-dimensional video data is captured video datadepicting a real-world event, the captured video data captured by aconfiguration of synchronous video cameras disposed at fixed positionsat the real-world event.
 3. The method of claim 2, further comprising:generating, by the virtual reality media provider system, a real-timevolumetric data stream representative of the dynamic volumetric modelsof the surfaces of the static object and the dynamic object; andproviding, by the virtual reality media provider system to a mediaplayer device and based on the real-time volumetric data stream, virtualreality media content representative of the real-world event asexperienced from a dynamically selectable viewpoint corresponding to anarbitrary location at the real-world event; wherein the dynamicallyselectable viewpoint is selectable by a user of the media player devicewhile the user is experiencing the real-world event using the mediaplayer device.
 4. The method of claim 2, wherein, along with beingdepicted in the two-dimensional video data, the surfaces of the staticobject and the dynamic object are spatially characterized by captureddepth data representative of the real-world event, the captured depthdata captured by a configuration of synchronous depth capture devicesdisposed, together with the synchronous video cameras, at the fixedpositions at the real-world event.
 5. The method of claim 4, wherein thegenerating of the dynamic volumetric model of the surface of the staticobject comprises: determining, based on the captured depth datacharacterizing the surface of the static object, that the static objectis statically located at a particular location at the real-world event,and that the static object is distinct from the dynamic object as thedynamic object periodically touches the static object; generating, basedon the determining that the static object is statically located at theparticular location and is distinct from the dynamic object, a wireframedepth model of the static object, the wireframe depth model representinga location in a three-dimensional space representative of the real-worldevent where the static object is positioned; and mapping, based on thetwo-dimensional video data depicting the static object, textures andcolors onto the wireframe depth model of the static object.
 6. Themethod of claim 4, wherein the generating of the dynamic volumetricmodel of the surface of the dynamic object comprises: determining, basedon the captured depth data characterizing the surface of the dynamicobject, that the dynamic object is continuously moving around at thereal-world event, and that the dynamic object is distinct from thestatic object as the dynamic object periodically touches the staticobject; generating, based on the determining that the dynamic object iscontinuously moving around at the real-world event and is distinct fromthe static object, a wireframe depth model of the dynamic object, thewireframe depth model representing positions on a path moved along bythe dynamic object through a three-dimensional space representative ofthe real-world event; and mapping, based on the two-dimensional videodata depicting the dynamic object, textures and colors onto thewireframe depth model of the dynamic object.
 7. The method of claim 2,wherein: the fixed positions at the real-world event where thesynchronous video cameras are disposed include fixed positions outsideof a stage space where the real-world event is taking place and that aredirected at the stage space; the static and dynamic objects at thereal-world event are located within the stage space; the fixed positionsat the real-world event where the synchronous video cameras are disposedfurther include a fixed position that is within the stage space and atwhich an outward-facing video camera is disposed, the outward-facingvideo camera configured to capture two-dimensional video data forsurfaces of external objects at the real-world event that are outside ofthe stage space; and the method further comprises differentiating, bythe virtual reality media provider system, the external objects in thetwo-dimensional video data from the static object and the dynamic objectin the two-dimensional video data, and generating, by the virtualreality media provider system and based on the differentiating of theexternal objects from the static object and the dynamic object, atwo-dimensional model representative of the surfaces of the externalobjects.
 8. The method of claim 1, wherein the differentiating of thestatic object from the dynamic object, the generating of the dynamicvolumetric models, and the updating and separately updating of thedynamic volumetric models are performed by the virtual reality mediaprovider system in real time.
 9. A system comprising: a memory storinginstructions; and a processor communicatively coupled to the memory andconfigured to execute the instructions to: differentiate a static objectdepicted in two-dimensional video data from a dynamic object depicted inthe two-dimensional video data, generate, based on the differentiatingof the static object from the dynamic object, a dynamic volumetric modelof a surface of the static object and a dynamic volumetric model of asurface of the dynamic object, update, with a lower regularity or on anas-needed basis, the dynamic volumetric model of the surface of thestatic object, and separately update, with a higher regularity that ishigher than the lower regularity and that keeps the dynamic volumetricmodel of the surface of the dynamic object up-to-date with what isoccurring in the two-dimensional video data, the dynamic volumetricmodel of the surface of the dynamic object.
 10. The system of claim 9,wherein the two-dimensional video data is captured video data depictinga real-world event, the captured video data captured by a configurationof synchronous video cameras disposed at fixed positions at thereal-world event.
 11. The system of claim 10, wherein the processor isfurther configured to execute the instructions to: generate a real-timevolumetric data stream representative of the dynamic volumetric modelsof the surfaces of the static object and the dynamic object; andprovide, to a media player device and based on the real-time volumetricdata stream, virtual reality media content representative of thereal-world event as experienced from a dynamically selectable viewpointcorresponding to an arbitrary location at the real-world event, thedynamically selectable viewpoint selectable by a user of the mediaplayer device while the user is experiencing the real-world event usingthe media player device.
 12. The system of claim 10, wherein, along withbeing depicted in the two-dimensional video data, the surfaces of thestatic object and the dynamic object are spatially characterized bycaptured depth data representative of the real-world event, the captureddepth data captured by a configuration of synchronous depth capturedevices disposed, together with the synchronous video cameras, at thefixed positions at the real-world event.
 13. The system of claim 12,wherein the generating of the dynamic volumetric model of the surface ofthe static object comprises: determining, based on the captured depthdata characterizing the surface of the static object, that the staticobject is statically located at a particular location at the real-worldevent, and that the static object is distinct from the dynamic object asthe dynamic object periodically touches the static object; generating,based on the determining that the static object is statically located atthe particular location and is distinct from the dynamic object, awireframe depth model of the static object, the wireframe depth modelrepresenting a location in a three-dimensional space representative ofthe real-world event where the static object is positioned; and mapping,based on the two-dimensional video data depicting the static object,textures and colors onto the wireframe depth model of the static object.14. The system of claim 12, wherein the generating of the dynamicvolumetric model of the surface of the dynamic object comprises:determining, based on the captured depth data characterizing the surfaceof the dynamic object, that the dynamic object is continuously movingaround at the real-world event, and that the dynamic object is distinctfrom the static object as the dynamic object periodically touches thestatic object; generating, based on the determining that the dynamicobject is continuously moving around at the real-world event and isdistinct from the static object, a wireframe depth model of the dynamicobject, the wireframe depth model representing positions on a path movedalong by the dynamic object through a three-dimensional spacerepresentative of the real-world event; and mapping, based on thetwo-dimensional video data depicting the dynamic object, textures andcolors onto the wireframe depth model of the dynamic object.
 15. Thesystem of claim 10, wherein: the fixed positions at the real-world eventwhere the synchronous video cameras are disposed include fixed positionsoutside of a stage space where the real-world event is taking place andthat are directed at the stage space; the static and dynamic objects atthe real-world event are located within the stage space; the fixedpositions at the real-world event where the synchronous video camerasare disposed further include a fixed position that is within the stagespace and at which an outward-facing video camera is disposed, theoutward-facing video camera configured to capture two-dimensional videodata for surfaces of external objects at the real-world event that areoutside of the stage space; and the processor is further configured toexecute the instructions to differentiate the external objects in thetwo-dimensional video data from the static object and the dynamic objectin the two-dimensional video data, and generate, based on thedifferentiating of the external objects from the static object and thedynamic object, a two-dimensional model representative of the surfacesof the external objects.
 16. The system of claim 9, wherein theprocessor is configured to differentiate the static object from thedynamic object, generate the dynamic volumetric models, and update andseparately update the dynamic volumetric models in real time.
 17. Anon-transitory computer-readable medium storing instructions that, whenexecuted, direct a processor of a computing device to: differentiate astatic object depicted in two-dimensional video data from a dynamicobject depicted in the two-dimensional video data; generate, based onthe differentiating of the static object from the dynamic object, adynamic volumetric model of a surface of the static object and a dynamicvolumetric model of a surface of the dynamic object; update, with alower regularity or on an as-needed basis, the dynamic volumetric modelof the surface of the static object; and separately update, with ahigher regularity that is higher than the lower regularity and thatkeeps the dynamic volumetric model of the surface of the dynamic objectup-to-date with what is occurring in the two-dimensional video data, thedynamic volumetric model of the surface of the dynamic object.
 18. Thenon-transitory computer-readable medium of claim 17, wherein thetwo-dimensional video data is captured video data depicting a real-worldevent, the captured video data captured by a configuration ofsynchronous video cameras disposed at fixed positions at the real-worldevent.
 19. The non-transitory computer-readable medium of claim 18,wherein the instructions further direct the processor to: generate areal-time volumetric data stream representative of the dynamicvolumetric models of the surfaces of the static object and the dynamicobject; and provide, to a media player device and based on the real-timevolumetric data stream, virtual reality media content representative ofthe real-world event as experienced from a dynamically selectableviewpoint corresponding to an arbitrary location at the real-worldevent, the dynamically selectable viewpoint selectable by a user of themedia player device while the user is experiencing the real-world eventusing the media player device.
 20. The non-transitory computer-readablemedium of claim 18, wherein, along with being depicted in thetwo-dimensional video data, the surfaces of the static object and thedynamic object are spatially characterized by captured depth datarepresentative of the real-world event, the captured depth data capturedby a configuration of synchronous depth capture devices disposed,together with the synchronous video cameras, at the fixed positions atthe real-world event.